Real - Time Multi - View Face Detection , Tracking , Pose Estimation , Alignment , and Recognition ( Updated Dec 1 , 2001 )

We have developed a system based on technologies resulting from our recent research in multi-view face detection and tracking [1, 2], pose estimation [3], alignment [4, 5, 6], and recognition [7]; where multi-view means out-of-plane rotations in [0Æ; 180Æ] left-right (90 corresponding to the frontal view) and in 30Æ up-down. The system, whose structure is shown in Fig.1, takes gray level static images or video as the input, without using color or motion information, and finds the locations, sizes, poses, and identities of faces in the input. It also includes internal feedback mechanisms for collecting and adding to the training sets new face examples for re-training the performing modules. Our objective has been to make these components work practically, fast and robustly. The methodologies used to achieve this for multi-view faces have evolved from our previous expensive nonlinear models [8, 9] to the current efficient linear models for face detection and alignment [1, 5], and from the previous unsupervised learning [10, 11, 12] to current supervised learning [3] for pose estimation. The mpeg clips for these demos can be downloaded at http://research.microsoft.com/ szli/Demos/CVPR01.html.